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Abstract 

This study uses a moving windows self-paced reading task to assess both text 
comprehension and processing time of authentic texts and these same texts simplified to 
beginning and intermediate levels. Forty-eight second language learners each read 9 texts 
(3 different authentic, beginning, and intennediate level texts). Repeated measures 
ANOVAs reported linear effects of text type on reading time (normalized for text length) 
and true/false comprehension scores indicating that beginning level texts were processed 
faster and were more comprehensible than intennediate level and authentic texts. The 
linear effect of text type on comprehension remained significant within an ANCOVA 
controlling for language proficiency (i.e., TOEFL scores), reading proficiency (i.e., 
Gates-MacGinitie scores), and background knowledge, but not for reading time. 
Implications of these findings for materials design, reading pedagogy, and text processing 
and comprehension are discussed. 

Keywords : second language reading, text simplification, computational linguistics, corpus 
linguistics, psycholinguistics, text processing, text comprehension. 


The role of spoken and written input is an important element of second language acquisition 
(Gass, 1988; Hatch, 1978) because it provides second language (L2) learners with language data 
from which to develop a lexicon, acquire the grammatical and syntactic structures of a language, 
and understand how the language conveys connections between ideas. The input that L2 learners 
receive is often simplified to make it more comprehensible. In spoken speech, L2 learners 
receive simplified input that is modified at the lexical, phonological, and syntactic level (Gaies, 
1983). Written L2 texts are also modified in a variety of ways to make the text more 
comprehensible. Such modifications generally occur at the syntactic and lexical level (Hill, 
1997). Written texts may also be simplified through a process of elaboration, which clarifies 
message content and structure through repetition of key ideas and the paraphrasing of difficult 
terms (Yano, Long, & Ross, 1994), although such elaboration can lead to decreased readability 
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(Long & Ross, 1993). 

A number of studies indicate that compared to authentic texts (i.e., unmodified texts that were 
originally created to fulfill a social purpose in a first language [LI] community; Little, Devitt, & 
Singleton, 1989), simplified texts lead to better text comprehension (Long & Ross, 1993; Oh, 
2001; Tweissi, 1998; Yano, et ah, 1994). However, many of these studies have not statistically 
controlled for potential linguistic differences between text conditions (Tweissi, 1998) or have 
relied on traditional readability formulas, which are limited in the number of linguistic features 
they measure (i.e., word and sentence length; Long & Ross, 1993; Yano et ah, 1994). 
Additionally, many of these previous studies did not control for language proficiency (Tweissi, 
1998; Yano et ah, 1994), reading proficiency (Long & Ross, 1993; Oh, 2001; Tweissi, 1998; 
Yano et al., 1994), or background knowledge (Long & Ross, 1993; Oh, 2001; Tweissi, 1998; 
Yano et al., 1994) when assessing text comprehension. Moreover, many of the studies focused 
only on comprehension and not on text processing speed (Oh, 2001; Tweissi, 1998; Yano et al., 
1994), which is a common metric of reading perfonnance in neuroscience, education, and 
psychology studies (Legge, 2006). Lastly, many previous studies have not compared 
comprehension at various levels of simplification (i.e., beginning and intennediate simplified 
texts) as compared to authentic texts (e.g., Long & Ross, 1993; Oh, 2001; Tweissi, 1998; Yano et 
al., 1994). 

This study addresses many of these issues within an experiment that recorded reading processing 
speeds and comprehension scores using a moving windows paradigm. The study examines 
reading processing and comprehension scores comparing texts simplified at the beginning level, 
at the intermediate level, and authentic texts. These texts differed linguistically at the level of 
lexical sophistication, syntactic complexity, and cohesion. The participants in this study were 
assessed for language proficiency, reading proficiency, and background knowledge, all of which 
were included as covariates in the statistical analyses. The primary research question that this 
proposed study seeks to answer is: 

1. Are there differences in text readability and comprehension for L2 readers among texts 
simplified to the beginning and intermediate level and authentic texts? 

2. To what degree do linguistic modifications in a text aid in text readability and 
comprehension for L2 readers when background knowledge, language proficiency, and 
reading proficiency are included as covariates? 

Text Simplification 

Texts simplified for L2 readers come in a variety of forms. They can be adapted or abridged 
versions of original texts, or texts written for teaching a specific grammar or linguistic form. 
However, all simplified texts share the same goal: increased comprehensibility and reduced 
cognitive load on the part of the L2 reader. The primary methods of attaining this 
comprehensibility are in the modification of the lexicon and the syntax (Hill, 1997). Generally, 
material developers have two approaches they can follow when simplifying a text: a structural 
approach or an intuitive approach (Allen, 2009). 

Intuitive approaches to text simplification are the most common of the two (Crossley, Allen, & 
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McNamara, 2012; Simensen, 1987). Following an intuitive approach, authors’ experiences as a 
language learner, language teacher, or materials developer (or any combination of these) guide 
the process of simplification. The intuition that is derived from such experiences allows the 
authors to rely on subjective approximations of what learners at a particular level should be able 
to read and comprehend (Allen, 2009). Such approximations obviate the need for structural 
manipulations. 

Under a structural approach to simplification, writers generally depend on pre-defined word and 
structure lists. Such approaches are commonly used in graded reader schemes that are aimed at 
advancing learners’ language acquisition through extensive reading. Another approach subsumed 
under the structural approach is text simplification guided by the use of traditional readability 
fonnulas. Traditional readability formulas are simple algorithms that measure text readability 
based on sentence length and word length. They have found success in predicting LI text 
readability, but have been widely criticized by discourse analysts (Davison & Kantor, 1982) and 
L2 researchers (Carrell, 1987; Crossley, Greenfield & McNamara, 2008) as being weak 
indicators of comprehensibility. Traditional readability fonnulas have also been demonstrated to 
be less effective at predicting text difficulty than readability formulas derived from indices that 
tap into cognitive processing (e.g., decoding, syntactic processing, and meaning construction; 
Crossley et ah, 2008). 

Text Simplification and Text Structure 

Until recently, the effects of text simplification on the actual linguistic features contained in 
simplified text were not clear, especially in comparison to authentic texts or in comparison of 
text simplification levels (i.e., differences in texts simplified at to beginning, intermediate, and 
advanced levels). This gap in the research was addressed in a series of studies by Crossley and 
his colleagues that compared differences in the linguistic features found in authentic and 
simplified texts (Crossley, Louwerse, McCarthy, & McNamara, 2007; Crossley & McNamara, 
2008) and differences between levels of simplification (Crossley, Allen, & McNamara, 2011; 
2012 ). 

In reference to differences between simplified and authentic texts, Crossley et al. (2011, 2012) 
used the computational tool Coh-Metrix (Graesser, McNamara, Louwerse, & Cai, 2004; 
McNamara & Graesser, 2012) to examine a wide range of linguistic features in authentic and 
simplified texts at the beginning and intennediate level. These studies have indicated that at the 
intermediate level, authentic texts are syntactically more complex and contain a greater density 
of logical connectors. Simplified texts, on the other hand, have significantly higher levels of 
cohesion (e.g., lexical co-reference and semantic overlap) and lower levels of lexical 
sophistication (e.g., word frequency, word hypemymy, word polysemy, and word familiarity). 
These findings are generally similar to those at the beginning level and suggest that simplified 
texts may be more cohesive than authentic texts and contain less sophisticated linguistic features. 
Thus, the findings from these studies support the notion that the process of simplification creates 
texts that contain more linguistic features that should lead to enhanced text comprehension and 
more rapid text processing. 

Crossley et al. (2011, 2012) used Coh-Metrix to examine differences between intuitively 
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simplified texts at the beginning, intermediate, and advanced levels. In the 2011 study, Crossley 
et al. investigated the predictive accuracy of readability formulas to distinguish among the three 
levels. They used two traditional readability fonnulas (Flesch Reading Ease, Flesch, 1948; 
Flesch-Kincaid Grade Level, Kincaid, Fishburne, Rogers, & Chissom, 1975) that measure word 
and sentence length and a more cognition-based readability formula (the Coh-Metrix L2 Reading 
Index, Crossley et al., 2008) that measures word frequency, syntactic similarity, and content 
word overlap. The results demonstrated that the Coh-Metrix L2 Readability Index predicted with 
greater precision the reading level of the texts as compared to traditional readability indices. The 
findings indicated that beginning level simplified texts contained more frequent words, higher 
syntactic similarity between sentences, and greater word overlap, all of which should lead to 
enhanced text comprehension and faster processing. The findings also raised concerns about the 
use of traditional readability formulas for assessing L2 text readability in that they were less 
predictive of simplification level and thus demonstrated lower correspondence with intuitive 
simplification approaches. 

In the 2012 study, Crossley et al. examined a variety of linguistic features related to lexical 
sophistication, syntactic complexity, and cohesion to predict simplified text level. The findings 
from this study indicated that beginning level simplified texts as compared to advanced texts are 
less complex lexically (e.g., contain lower lexical diversity, more frequent words, more familiar 
words, and more concrete words) and syntactically (e.g., have greater syntactic similarity and 
fewer words before the main verb). In addition, beginning level simplified texts contain a greater 
number of cohesive cues (e.g., more given information, greater semantic co-referentiality, and 
more noun overlap). These findings imply that beginning level simplified texts should be easier 
to process and lead to enhanced comprehension than advanced level simplified texts. 

Simplification and Text Comprehension 

Material designers and researchers have considerable interest in the benefits of text 
simplification for L2 learners. Generally, the effects of text simplification are measured in terms 
of increased text comprehension, with supporters of text simplification arguing that the process 
of simplification increases the reader’s ability to understand and interact with a text (Goodman & 
Freeman, 1993). Detractors, on the other hand, maintain that the removal of linguistic fonns in 
favor of more simplified and frequent forms deny learners the opportunity to interact with and 
acquire the natural forms of language (Long & Ross, 1993). Generalizations about 
comprehension effects that result from authentic or simplified texts are difficult because of 
differences in research designs. However, the research that has been conducted generally 
supports the facilitative effects of text simplification for text comprehension. 

For instance, Long and Ross (1993) found that L2 students who read texts linguistically 
simplified using traditional readability formulas scored significantly higher on multiple-choice 
items intended to assess comprehension than did those that read authentic texts. Long and Ross 
also found a strong relationship between readers’ English proficiency level and reading 
comprehension scores with more proficient readers exhibiting better comprehension. This 
finding was supported in a follow up study (Yano et al., 1994) using the same texts. This study 
similarly demonstrated that the simplified texts increased text comprehension in comparison to 
the authentic texts. More recent studies by Tweissi (1998) and Oh (2001) also found that 
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simplification positively affected L2 students’ overall reading comprehension. In addition, 
Tweissi reported differences in comprehension comparing texts modified both lexically and 
syntactically as compared to texts just modified lexically (with lexical modifications only 
leading to increased comprehension). This finding was interpreted as indicating that excessive 
simplification may reduce reading comprehension. However, Tweissi did not statistically control 
for linguistic differences in the text conditions, rendering interpretations of this finding 
potentially spurious. Similar to the studies reported by Tweissi, Long and Ross, and Yano et al., 
the study reported by Oh also found that simplified texts showed significant comprehension 
gains over authentic texts for high proficiency readers, but not for low proficiency readers. 
However, like Tweissi, Oh did not statistically control for linguistic differences between the text 
conditions. 


Methods 

As discussed in the previous section, a number of studies indicate that text simplification leads to 
increased text comprehension when compared to authentic texts. However, many of these studies 
did not statistically control for linguistic differences in text conditions (the exceptions being 
those studies that relied on readability fonnulas). Additionally, many of the studies did not 
control for participants’ language proficiency, reading proficiency, or background knowledge 
when assessing text comprehension. Previous studies have also focused only on comprehension 
and not on text processing speed, which is an important indicator of reading perfonnance (Legge, 
2006). Lastly, previous studies have compared simplified text to authentic texts but have not 
controlled the level of simplification. This study addresses these limitations by using a self-paced 
moving window reading task to assess text processing speeds and true/false comprehension 
questions to assess text based levels of comprehension for authentic texts and texts simplified to 
the beginning and intennediate level. In addition, language proficiency, reading proficiency, and 
background knowledge scores were collected from the participants in this study. 

Participants 

Forty-eight non-native speakers of English enrolled at the Instituto Tecnologico y de Estudios 
Superiores de Monterrey (ITESM) campus in San Luis Potosi, Mexico participated in this study. 
All participants were native speakers of Spanish and were studying at either the high school or 
college level. Of these 48 participants, 19 were female and 29 were male. All participants had 
normal or corrected to normal vision. The participants ranged in age from 15 to 24. All 
participants had taken a paper-based institutional TOEFL 1 month before the data collection. The 
average TOEFL score for the participants was 520. Descriptive statistics for the participants are 
presented in Table 1. 


Table 1. Descriptive statistics for participants in study 


Item 

Minimum 

Maximum 

Mean 

SD 

Age 

15 

24 

17.71 

2.153 

Grade level 

10 

13 

11.58 

1.164 

Grade point average (100 scale) 

73 

97 

84.81 

6.271 

TOEFL scores 

420 

597 

519.60 

30.741 
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Procedure 

Data collection occurred in three different sessions. The first session involved an on-line 
questionnaire that collected demographic information for the participants followed by a 
background knowledge survey that assessed participants’ knowledge of the topics covered with 
the reading passages. One week later, in the second session, the participants were administered 
the Gates-MacGinitie reading proficiency test (GMRT, MacGinitie & MacGinitie, 1989). On the 
following day, during the third session, the students participated in an on-line reading experiment 
that assessed their ability to read simplified and authentic texts using a self-paced, non- 
cumulative, moving window reading task similar to that used by Just, Carpenter, and Woolley 
(1982). Comprehension of these texts was assessed using true/false questions. Moving window 
reading tasks are common in studies of second language syntactic acquisition research (SLA, 
Felser, Roberts, Gross, & Marinis, 2003; Juffs & Harrington, 1995), but not L2 reading research 
to date. Moving windows tasks simulate eye movement data, but at a fraction of the cost (Just, 
Carpenter, & Woolley, 1982). 

Three text groupings were developed for the moving window reading task such that each 
grouping included three authentic texts, three texts simplified to the intermediate level, and three 
texts simplified to the beginning level (n = 9). There was no overlap between the texts in each 
grouping and the texts in each grouping were on different topics. The texts were presented in 
random order. Participants were randomly, but evenly assigned to a grouping so that each text at 
each level and each text was read by at least 16 participants. Thus, each participant read nine 
texts (three beginning level simplified texts, three intermediate level simplified texts, and three 
authentic texts) on nine different topics. 

Each text was presented one word at a time and the participants advanced through a text by 
pushing the spacebar on a computer keyboard. The words were presented sequentially and in the 
same location as in a normal text. Participants were not allowed to revisit text that had already 
been read. Such an approach allows a processing time measure (i.e., response times between 
spacebar presses) to be associated with each individual word (similar to word fixation rates). 
Participants were given instructions on the tasks and a practice trial to familiarize themselves 
with the task. Comparisons between self-paced reading tasks and nonnal reading tasks have 
demonstrated similar results across a number of studies (Juola, Ward, & McNamara, 1982; 

Rubin & Turano, 1992); however, this is not always the case (cf. Kennedy & Murray, 1984; 
Magliano, Graesser, Eymard, Haberlandt, & Gholson, 1993). In addition, self-paced reading 
tasks such as moving windows tasks slow down the reading time (Rayner, 1998). 

When participants reached the end of the text, they were prompted to answer yes/no 
comprehension questions about the text they had just read (see Comprehension Question section 
below for more detail). The experiment was developed using E-Prime software. A font size of 14 
was selected to ensure that visual factors did not affect reading speed (Legge, Rubin, Pelli, & 
Schleske, 1985). 
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Materials 

Texts. The reading samples used for this study were selected from a corpus of 100 simplified 
news texts. Thus, unlike previous studies (Long & Ross, 1993; Oh, 2001; Tweissi, 1998; Yano et 
ah, 1994), we did not modify the texts specifically for this study, but rather selected pre-existing 
texts modified by experts (see below for details). The texts were taken from an English teaching 
website (www.onestopenglish.com). The website offers simplified news texts and accompanying 
learning activities. The news texts were originally taken from the Guardian Weekly, a British- 
based publication with a wide international readership. The articles in the corpus were originally 
selected by the website editors for their topicality and interest value and typically center on 
world affairs. The texts are, therefore, non-academic in nature. 

The news texts were simplified by a small, independent team of authors, into three levels of 
simplification: advanced, intermediate and beginning. The method of simplification employed by 
the authors was intuitive, that is, without recourse to word lists, structural grading schemes, or 
readability formulas (Allen, 2009). As reported by Allen, the authors did provide certain 
indications regarding their approach to simplification. First, the authors followed the motto 
‘grade the task, not the text,’ showing a tendency to only simplify when absolutely necessary. 
Second, a number of general strategies were employed, such as modifying idiomatic language at 
the intennediate level and removing it completely from the elementary level, while removing all 
passive structures and phrasal verbs from elementary level texts. The texts have been used in a 
number of studies examining text simplification in L2 reading (Allen, 2009; Crossley et ah, 2011, 
2012 ). 

From this corpus, we selected the initial paragraph from nine texts to use in the moving windows 
experiment. For each text we had three versions: the authentic text, a text simplified to the 
intermediate level, and a text simplified to the beginning level. We did not select texts from the 
advanced level because these texts were generally unmodified from the authentic texts (Allen, 
2009). All texts contained the same main propositional information. While rarefy necessary, we 
changed all spelling in the texts from British English to American English, which was the 
English of instruction for our participants. 

In the process of text selection, we selected only those texts that differed in linguistic features 
related to L2 text simplification, comprehension, and readability. We examined the linguistic 
features in the text by first collecting linguistic infonnation for each text using the computational 
tool Coh-Metrix (Graesser et ah, 2004; McNamara & Graesser, 2012). We then used the 
linguistic information from each text to assess differences between levels using a Multivariate 
Analysis of Variance (MANOVA). The linguistic features we selected were related to meaning 
construction (i.e., cohesion), lexical recognition (i.e., lexical sophistication), and syntactic 
parsing (i.e., syntactic complexity). These features were selected from previous Coh-Metrix 
research that examined linguistic differences between simplified and authentic texts (Crossley et 
al., 2007; Crossley & McNamara, 2008), differences between levels of simplified texts (Crossley 
et al., 2012), and L2 text comprehensibility (Crossley et al., 2008). All indices demonstrated 
significant differences for the selected features between the text levels (see Table 2). The 
selected indices are discussed below. We also ensured that text conditions did not significantly 
differ based on number of words. 
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Table 2. Multivariate results for linguistic differences between text levels 


Linguistic features 

Beginning level 
text mean (SD) 

Intermediate level 
text mean (SD) 

Authentic text 
mean (SD) 

/ 

P 

hp2 

Noun overlap 

0.55 

(0.13) 

0.27 

(0.13) 

0.15 

(0.14) 

21.371 

<.001 

0.640 

Lexical diversity D 
CELEX content 

65.89 

(14.85) 

91.00 

(14.93) 

112.67 

(24.65) 

14.081 

<.001 

0.540 

word frequency 
Sentence syntax 

2.43 

(0.17) 

2.20 

(0.20) 

2.01 

(0.16) 

12.449 

<.001 

0.509 

similarity 

0.13 

(0.03) 

0.10 

(0.03) 

0.07 

(0.03) 

8.707 

<.001 

0.420 

Word familiarity 
Word 

580.55 

(8.97) 

568.74 

(10.70) 

563.16 

(9.63) 

7.389 

<.010 

0.381 

Meaningfulness 
Number of causal 

368.12 

(14.06) 

352.70 

(14.14) 

346.89 

(15.82) 

5.989 

<.010 

0.333 

verbs and particles 
LSA Sentence to 

44.12 

(18.84) 

32.87 

(12.65) 

23.70 

(8.84) 

4.761 

<.050 

0.284 

Sentence mean 

0.23 

(0.07) 

0.18 

(0.05) 

0.14 

(0.07) 

4.352 

<.050 

0.266 

Number of words 

150.11 

(29.80) 

125.22 

(21.93) 

128.89 

(28.25) 

2.250 

>.050 

0.158 


Cohesion. Cohesion is an important feature of text readability that is strongly related to enhanced 
comprehension and processing (Gernsbacher, 1997; McNamara, Kintsch, Songer, & Kintsch, 
1996). Coh-Metrix calculates a number of indices related to text cohesion including word 
overlap, semantic similarity, and causality. Word overlap indices measure how often words 
overlap between adjacent sentences. Semantic similarity indices use Latent Semantic Analysis 
(Landauer, McNamara, Dennis, & Kintsch, 2007) to assess similarity between words at the 
sentence and paragraph level. Word and semantic overlap indices help to facilitate meaning 
construction and improve text comprehension and reading speed (Douglas, 1981; Kintsch & van 
Dijk, 1978; Rashotte & Torgesen, 1985). Greater lexical overlap also leads to enhanced text 
readability for L2 readers (Crossley et ah, 2008). Causal indices in Coh-Metrix are measured by 
calculating the ratio of causal verbs to causal particles in a text, which relates to the conveyance 
of causal content and causal relations between events and actions. Past studies have 
demonstrated that beginning level simplified texts have greater word overlap, greater semantic 
similarity, and less causality than intennediate and advanced simplified texts (Crossley et al., 
2012) and authentic texts (Crossley & McNamara, 2008). 

Lexical sophistication. The level of lexical sophistication found in a text also relates to text 
processing and comprehension and is an important feature of word decoding (Just & Carpenter, 
1987; Perfetti, 1985; Rayner & Pollatsek, 1994). In reference to lexical sophistication, Coh- 
Metrix calculates the word frequency, word familiarity, word meaningfulness, and lexical 
diversity for the words in a text. Coh-Metrix calculates word frequency using the CELEX 
database (Baayen, Piepenbrock, & Gulikers, 1995), which consists of word frequencies taken 
from the 17.9 million word COBUILD corpus. Coh-Metrix calculates word familiarity and word 
meaningfulness using the MRC Psycholinguistic Database (Coltheart, 1981). Word familiarity 
indices assess the likelihood of word exposure while word meaningfulness indices measure how 
strongly words associate with other words. Coh-Metrix calculates lexical diversity using a 
variety of indices that control for text length including D (Malvern, Richards, Chipere, & Duran, 
2004), which was used in this study. Word frequency is a strong predictor of L2 reading 
performance (Crossley et al., 2008) while word familiarity, meaningfulness, and lexical diversity 
are important elements of the text simplification process (Crossley et al., 2012; Crossley & 
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McNamara, 2008). More details on these indices can be found in McCarthy and Jarvis (2010) 
and Salsbury, Crossley, and McNamara (2011). 

Syntactic Complexity. Syntactic complexity indices relates to the difficulty of syntactic parsing 
(Just & Carpenter, 1987; Rayner & Pollatsek, 1994), which is an important element of text 
readability and comprehension. From Coh-Metrix, we selected an index that measures semantic 
similarity by assessing the unifonnity and consistency of parallel syntactic constructions in text. 
More uniform syntactic constructions result in less complex syntax that is easier for the L2 
reader to process (Crossley et ah, 2008). Greater syntactic similarity is also a feature of 
beginning level L2 simplified text (Crossley et ah, 2012). 

Number of words in the text. We also included a measure of text length because text length can 
correlate to the number of propositions contained in a text. The number of propositions found in 
text relates to text comprehension because multiple propositions strain working memory 
(Kintsch & Keenan, 1973). We used this measure to ensure that the conditions did not differ as a 
function of text length. 

Background knowledge tests. Background knowledge is an important predictor of readability and 
text comprehension (McNamara et ah, 1996; Shapiro, 2004). Thus, we developed an assessment 
of background knowledge specific to the reading texts used in this study. Following Bellissens, 
Jeuniaux, Duran, and McNamara (2010), we developed prior knowledge questions for each of 
the sets of texts. These questions broadly covered the main themes shared in the same set of 
beginning simplified, intermediate simplified, and authentic texts. The questions were multiple- 
choice questions that included the correct answer and three distracters. The distracters were near- 
miss (incorrect in general), thematic (same theme but incorrect), and unrelated (different theme 
and incorrect). We developed five text-based questions for each text (N = 45) where the question 
and answer were in the same sentence. We piloted these questions with 25 undergraduate 
students in order to examine item performance. Twenty-seven questions (three for each set of 
texts) were selected based on the item performance such that no questions had either a ceiling (r 
> .900) or a floor effect (r < .250). We used these 27 questions to assess the background 
knowledge of the participants for the text topics used in this experiment. 

Reading proficiency tests. Participants in this experiment were administered the Gates- 
MacGinitie Reading Test (GMRT, MacGinitie & MacGinitie, 1989), which is a multiple-choice 
reading test that lasts 55 minutes. The participants were all administered the Level 10/12 GMRT. 
The tests were scored by the researchers. Controlling for reading proficiency is an important 
element of reading research, especially research that considers reading time because word 
fixation will decrease as a function of increasing reading skill (Buswell, 1922). 

Comprehension questions. After participants finished reading a text in the moving windows 
experiment, they answered four true/false questions that corresponded to the main ideas and 
important details of the text. The comprehension questions associated with each text were 
identical regardless of level (i.e., text simplified to beginning or intermediate level and authentic 
texts). In this way, the questions were text specific. Such comprehension questions are a 
common approach to assessing reading for understanding (Carver, 1990). 
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Statistical Analyses. Statistical analyses of the data (i.e., reading times and comprehension scores) 
were conducted to examine the effects of linguistic simplification on text readability and 
comprehension. Analyses of variance (ANOVAs) followed by analyses of covariance 
(ANCOVA) were conducted to examine differences in reading times and comprehension scores 
between beginning, intermediate, and authentic texts alone and by using TOEFL, background 
knowledge, and reading proficiency (GMRT) scores as covariates. 


Results 

Correlations Between Covariates 

Pearson correlations were conducted among the covariates selected for this analysis (i.e., TOEFL, 
background knowledge, and GMRT scores) to examine potential multicollinearity. None of the 
covariates demonstrated strong multicollinearity (defined as r > .70) with each other (see Table 
3). 


Table 3. Correlations between selected covariates 



Background knowledge score 

GMRT score 

TOEFL score 

-0.035 

0.559 

Background knowledge score 


0.068 


Reading Time ANOVA 

A within-subjects ANOVA revealed a significant effect of text level (beginning, intermediate, 
authentic texts) on reading time (normed for text length and reported as milliseconds per word), 

F (2, 94) = 3.688, p < .050, r| p 2 = .073. A significant linear trend, F(l, 47) = 6.523, p < .001, r| p 2 
= .122, confirmed the expectation that texts simplified to the beginning level result in the shortest 
reading times followed by texts simplified to intennediate and authentic levels (see Table 4). 
Follow-up pairwise comparisons were conducted to evaluate differences among the means. 
Beginning level texts took less time to read than authentic texts ip < .050), but not intermediate 
texts. No differences in reading times were found between authentic texts and intennediate texts. 


Table 4. Descriptive statistics for reading time (ms per word): Mean (SD) 


Text level 

Reading time 

Beginning 

603.20 (185.09) 

Intermediate 

624.00(166.42) 

Authentic 

635.44(162.49) 


Tests for Homogeneity of Regression Slopes: Reading Time 

Prior to conducting an ANCOVA, regression slopes for each of the covariates were plotted 
against the reading times for the participants by condition. The regression slopes were used to 
assess homogeneity between the covariates in the prediction of the dependent variable (i.e., 
reading time scores) across the conditions. If the regression slopes for the conditions are similar 
(i.e., demonstrate homogeneity in their slopes), then the dependent variable likely does not vary 
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as a function of the covariates and the covariates can be used in the subsequent ANCOVA. If the 
regression slopes for the conditions are dissimilar (i.e., demonstrate heterogeneity in their slopes), 
then the differences in the dependent variable among conditions likely vary as a function of the 
covariate and the results from the ANCOVA may not be meaningful because the findings will 
not represent all the conditions. The regression plots for the reading time covariates showed 
homogeneity across the conditions (see Figure 1) and thus all covariates were used in the 


ANCOVA. 



TOEFL Scores 



condition 
Beginning 
Intermediate 
O Authentic 

Beginning: R 2 Linear = 0.013 
Intermediate: R 2 Linear » 0.004 
Authentic: R 2 Linear = 0.048 
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condition 
Beginning 
'• Intermediate 
O Authentic 

Beginning: R 2 Linear = 0.219 
Intermediate: R 2 Linear = 0.267 
Authentic: R 2 Linear = 0.186 


Figure 1 . Regression slopes for reading time and covariates 
Reading Time ANCOVA 


The within-subjects ANCOVA on reading time including the covariates (i.e., TOEFL, GMRT, 
background knowledge) was not significant, F (2, 88) = 1.373, p > .050, r| p 2 = .030. The effects 
of language proficiency (F < 2) and background knowledge [F (1,44)=2.002] were not 
significant. However, reading proficiency as measured by GMRT had a substantial relation with 
participants’ reading times, F(l, 44) = 14.779,/? < .001, r| p 2 = .251. Therefore, when reading 
proficiency was entered as a covariate, the differences in reading time as a function of text level 
were no longer significant. 


Comprehension Score ANOVA 


The within-subjects ANOVA revealed a significant effect of text level (beginning, intermediate, 
authentic texts) on comprehension scores, F (2, 94) = 5.732, p< .001, r| p 2 = .109. A significant 
linear trend, F(l, 47) = 9.093,/? < .001, r| p 2 = .162, confirmed expectations that the beginning 
level texts produce the highest comprehension scores followed by intermediate and authentic 
texts (see Table 5). Follow-up pairwise comparisons demonstrated that beginning simplified 
texts result in higher comprehension scores than intermediate simplified texts (/? < .050) and 
authentic texts (p < .010), with no differences between intermediate and authentic texts {p >. 
050). 
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Table 5. Descriptive statistics for comprehension scores: Mean (SD) 


Text level 

Comprehension score 

Beginning 

.81 (.14) 

Intermediate 

.76 (.13) 

Authentic 

.72 (.16) 


Tests for Homogeneity of Regression Slopes: Comprehension Scores 

Regression slopes were plotted for each of the covariates against the comprehension scores for 
the participants by condition. The regression plots for the background scores showed 
heterogeneity across the conditions suggesting that differences in the dependent variable among 
the conditions varied as a function of the covariate (see Figure 2). Specifically, Figure 2 
illustrates that background knowledge had a greater effect (i.e., steeper slope) for authentic texts 
than for simplified or intermediate texts. Thus, this covariate was not used in the ANCOVA. The 
TOEFL scores and reading proficiency scores demonstrated similarities across the conditions 
and were thus included in the ANCOVA 



Conditon 
Beginning 
Intermediate 
O Authentic 

Beginning: R 2 Linear = 0.021 
Intermediate: R 2 Linear = 0.006 
Authentic: R 2 Linear = 0.155 
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Conditon 
Beginning 
Intermediate 
O Authentic 

Beginning: R 2 Linear = 
Intermediate: R 2 Linear 
Authentic: R 2 Linear = 


0.003 
= 0.007 
0.069 



Figure 2. Regression slopes for comprehension scores and covariates 


Reading in a Foreign Language 26(1) 











Crossley, Yang, & McNamara: What’s so simple about simplified texts? 


106 



Text level 

Figure 3. Reading comprehension as a function of TOEFL score (high, low) 
Comprehension Score ANCOVA 

The within-subjects ANCOVA on comprehension scores including the covariates (i.e., TOEFL, 
GMRT) was significant, F (2, 90) = 4.661 ,p< .050, r| p 2 = .094, confirming a significant linear 
effect of text level, F (1, 45) = 4.927, p < .050, r| p 2 = .099. Therefore, when the effects of 
language proficiency as measured by TOEFL (F<1) and reading proficiency as measured by 
GMRT [FIT, 45) = 6.38 6,p< .050, r| p 2 = .124] were removed statistically, the effects oftext 
level on comprehension remain significant. 

In addition, there was a significant interaction between text level and TOEFL score, F (2, 90) = 
4.790,/? < .050, rjp 2 = .096. To illustrate this interaction, participants were categorized as high or 
low TOEFL using a mean split (TOEFL M = 520). Figure 3 displays comprehension scores as a 
function of TOEFL score group (high, low) and text level. Whereas there was little effect of 
language proficiency according to TOEFL for the simple and intermediate texts (both F < 2), 
there was a significant effect for authentic texts, F (1, 46) = 5.065,/? < .050, r| p 2 = .100, 
demonstrating that authentic texts were comprehended better by participants with higher TOEFL 
scores than by participants with lower TOEFL scores (see Figure 3). 


Discussion 

A primary goal of this study was to examine the effects of text simplification on reading times 
and comprehension scores for L2 readers. We were also interested in assessing these differences 
while also statistically accounting for participants’ reading proficiency, language proficiency, 
and background knowledge. 
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As expected, we found a linear trend such that authentic texts were read more slowly than were 
simplified texts. The reported effect size indicated that simplification explained about 12% of the 
variance in reading time. The reading time differences assumedly emerge from the linguistic 
differences between the texts, where simplified texts are characterized by more familiar words 
and text that is more cohesive and syntactically less complex. In essence, simplified text can be 
characterized as being more readable and as using language that is more accessible. 

One qualification to the reading time results arises from the ANCOVA results. When reading 
proficiency was included as a covariate, the differences in reading time as a function of text level 
were no longer significant. Hence, reading proficiency tended to drive the reading times for the 
L2 readers, more so than the nature of the text. These findings collectively indicate that for these 
L2 readers there are significant effects of text simplification on reading time (especially between 
beginning simplified texts and authentic texts), but the variance associated with reading times is 
better explained by individual differences in reading proficiency (which explain about 25% of 
the variance in reading time). These findings are not surprising in that reading proficiency 
measures such as GMRT were explicitly developed to detect differences in reading processes 
and outcomes, such as reading time, whereas the differences between texts as a function of 
simplification for L2 readers are likely more subtle. In addition, text simplification is typically 
intended to facilitate comprehension as opposed to reading speed. 

Similar to the effects on reading time, there was a linear effect of text simplification on 
comprehension. These results confirm that the beginning level simplified texts were 
comprehended better than authentic texts. Hence the ease in processing was not traded off with 
comprehension: simplified texts were read more quickly and they were better understood. 

This effect of text simplification on comprehension remained when reading proficiency and 
language proficiency were included as covariates. However, one qualification to this finding 
emerged from an interaction between text level and language proficiency (i.e., TOEFL scores). 
Namely, the effect of text simplification on comprehension crucially depended on language 
proficiency. L2 readers with stronger language proficiency were better able to cope with the 
linguistic challenges of authentic text. By contrast, participants with lower English proficiency 
were significantly challenged by the authentic texts. Hence, the disadvantages of the authentic 
texts were augmented for those L2 readers with lower language abilities. 

Authentic texts were also particularly challenging for low-knowledge readers, but appeared to be 
more beneficial to the readers with higher background knowledge. Looking at the 
comprehension score regression slopes in Figure 2, we see that comprehension differences as a 
function of text level further depended on the participants’ background knowledge. Specifically, 
participants with higher background knowledge better comprehended authentic texts than those 
with less background knowledge. At the same time, high knowledge participants comprehended 
beginning simplified texts to a lesser degree than low background knowledge readers. These 
findings mirror reverse cohesion effects reported with LI readers (McNamara, 2001; McNamara 
& Kintsch, 1996; McNamara et al., 1996; O’Reilly & McNamara, 2007). Accordingly, low 
background knowledge readers show better understanding of high cohesion (e.g., simplified) 
texts because the cohesion cues help the readers bridge gaps in their background knowledge. By 
contrast, high knowledge readers show comprehension gains for texts with lower cohesion (e.g., 
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authentic texts). The predominate explanations for this reverse cohesion effect is that the lower 
cohesion texts induce the high knowledge readers to generate inferences that connect ideas in the 
text and also connect the text with prior knowledge. This interpretation is bolstered in the current 
study by the reading time results. Looking at the reading time regression slopes in Figure 1, we 
see that participants with more background knowledge take longer to read authentic texts than 
participants with less background knowledge. This increase in reading time is likely attributable 
to the increased effort needed to bridge cohesion gaps. The gaps induce inference generation 
(which takes more time), and the high knowledge readers are more able to engage in these gap- 
filling processes. In turn, these readers showed higher comprehension for these texts, because 
they were more actively engaged in the comprehension process. Nonetheless, the potential 
advantages of authentic texts are offset by their negative effects for lower knowledge readers, 
who depend on the higher cohesion in the simplified texts to scaffold their understanding. 

Overall, the comprehension findings provide support for previous studies that have indicated that 
text simplification enhance text comprehension (Long & Ross, 1993; Oh, 2001; Tweissi, 1998; 
Yano et al., 1994). The reported effect size from the comprehension ANOVA indicated that 
simplification explained about 16% of the variance in comprehension scores. This relation 
between text simplification and comprehension seems to hold when reading proficiency and 
language proficiency are held constant with the reported effect size indicating that simplification 
explained about 10% of the variance in comprehension scores in the ANCOVA. Moreover, the 
advantages of simplified text are amplified for readers with lower language proficiency (low 
TOEFL scores). However, it is not unambiguously the case that simplifying a text will lead to 
better comprehension. The background knowledge of the reader is an important consideration. If 
a reader has higher background knowledge of the text’s topic, authentic text enhances inference 
generation, which in turn improves comprehension. Conversely, if a reader has low background 
knowledge, beginning level texts enhance comprehension in comparison to authentic texts. 

Pedagogically, these findings have important implications for L2 reading, classroom applications, 
and L2 acquisition. These results collectively confirm what might seem obvious post hoc. 
Language teachers selecting texts should consider the aims of the reading task and the reading 
proficiency of the students. Clearly, simplification at the beginning level is beneficial for L2 
readers’ text comprehension regardless of reading and language proficiency. Nonetheless, 
depending on the pedagogical objectives, the reading instructor may do better to consider the 
attributes of the students rather than to rely solely on text simplification. For example, high 
ability L2 readers (with higher language proficiency or higher background knowledge) seemed 
less intimidated by the authentic texts and potentially gained from the active processing induced 
by the lower cohesion. Hence, authentic texts may be particularly useful if the content is 
relatively familiar to the students and when the instructional goal is to emphasize active 
comprehension of the material. 

Interestingly, it is important to note that across all of the results the intennediate texts sat 
squarely in the middle, with few differences between either authentic or simplified texts. 
Simplification seems to make little difference for comprehension at the intermediate level for L2 
readers and leads to comprehension scores that are similar to those found for authentic texts. This 
(null) result implies that the intermediate texts offer a compromise between the two, and this 
compromise may have few negative consequences. 
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Linguistically, these findings demonstrate that texts that are more cohesive (i.e., more noun 
overlap, more causality, and more semantic similarity) can lead to increased text comprehension, 
further indicating that cohesive features that link sections of text together help to promote 
comprehension. Lexically, texts that have lower lexical diversity, more infrequent words, more 
familiar words, and more meaningful words can enhance comprehension, likely as a result of 
simplified propositional information. Lastly, texts that are syntactically simpler also facilitate 
text comprehension. This benefit is likely attributable to the greater ease in parsing the texts 
which opens the door to understanding them at a more global level. Such claims clearly 
necessitate a more fine-grained analysis, but the findings of this study nevertheless bolster claims 
that linguistic modifications at the cohesion, lexical, and syntactic level do increase text 
comprehension for L2 readers. 


Conclusion 

This study generally supports the use of simplified texts for L2 readers because such texts 
promote reading comprehension. However, text simplification does not always lead to decreased 
reading times or enhanced comprehension. These findings indicate that the use of simplified 
texts for L2 learning comes with caveats. First, text simplification does appear to lead to 
increased text comprehension, but may only do so if the L2 reader does not have strong 
background knowledge of the topic. In addition, the use of authentic texts seems to benefit 
advanced L2 learners to a greater degree than beginning L2 learners. Second, text simplification 
can lead to decreased reading times, but reading ability is potentially a more important 
consideration in that regard. 

The research paradigm used in this study has provided valuable infonnation about text 
processing and reading speed. However, it is not without its drawbacks. Assessing reading 
comprehension and cognitive processing in general is complex matter and no one technique can 
address the reading process as a whole. Thus follow-up studies using a variety of other research 
techniques (i.e., eye tracking and event-related brain potentials) would prove beneficial (Ditman, 
Holcomb, & Kuperberg, 2007; Rayner, 1998). Such techniques would be helpful in reassessing 
the reading time results reported in this study because moving windows techniques, such as the 
one used in this study, do not allow for multiple fixations of a single word or the skipping of 
words, both of which are common for readers. Also, moving windows techniques have been 
criticized because the task slows down the reading rate (i.e., the fingers move slower than the 
eyes); hence, generalizations taken from this study should be taken with caution (Rayner, 1998). 
As is always the case following an experimental study, follow-up studies are clearly warranted. 
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